bun test: decide Jest Cause: lines with the snapshot printer's instanceof Error walk - #48046

Two questions for a maintainer

1. A message-only entry of an error that has causes, in a Jest-format file whose keys are in order. bun test: Vitest compatibility #44833 compares such a file with the text of Jest only, because it takes a sorted file to hold no entry from Bun (may_hold_entries_of_bun, src/runtime/test_runner/snapshot.rs:221). Bun up to 1.4 does leave such entries there: in a file with one entry, or when the tests run in the order of their names. Jest 29 and older wrote the same text, under the first line // Jest Snapshot v1, https://goo.gl/fbAQLP. Should these entries keep passing through the comparison with an older Bun's text? Then an "outer" that Jest 30 wrote also passes after the code starts to attach a cause, where Jest 30 fails it. Or is the verdict of Jest 30, and one run of --update-snapshots, the intended outcome? This PR does not change that rule, and main has the same failure for a native Error or a string as the cause. A rule that goes by the goo.gl first line does not cover an entry that Bun wrote into a file with the new first line. The numbers are under "Message-only entries" and "Files of Jest 29" below.
2. Jest's cause instanceof Error asks a test-defined Error[Symbol.hasInstance] and the current globalThis.Error. This PR asks neither, like SnapshotFormat__kindOf. If the hook must be asked, the change is inside the 5-line export SnapshotFormat__inheritsErrorPrototype.

Who hits this

• No user reported it. A probe found it while bun:test: toThrowErrorMatchingSnapshot stores the message of a thrown object that is not an Error #47999 was in work. The loop came with bun test: Vitest compatibility #44833, which is in no release.
• A released behaviour does change: see "Message-only entries" below. An earlier version of this description said that none does. That was wrong.
• Real causes of this kind: jwt.verify() of jsonwebtoken 9.0.2 throws a JsonWebTokenError, and rxjs 7.8.2 has EmptyError and TimeoutError. They are instances of ES5 classes: instanceof Error is true and Error.isError() is false. Jest 30.5.2 stores Cause: invalid token for the first. On main 0 of these 3 rows pass. With this PR 3 of 3 pass.

What Jest does (jest-snapshot 30.5.2, _toThrowErrorMatchingSnapshot)

let message = error.message;
while ('cause' in error) {
  error = error.cause;
  if (isError(error) || error instanceof Error) {
    message += `\nCause: ${error.message}`;
  } else {
    if (typeof error === 'string') message += `\nCause: ${error}`;
    break;
  }
}

isError is the one of jest-util: Error.isError, or util.types.isNativeError where that does not exist. It is not the isError of expect-utils, which reads Symbol.toStringTag.

Rows that Jest 30.5.2 wrote

The same counts hold with --isolate, --isolate=modules and --parallel=2. With an empty .snap, Bun writes the corpus entries byte for byte as Jest did.

Where Bun differs from Jest after this PR

• One realm. Jest on Node runs tests in a vm context. An error that Node made (an fs error, a DOMException, an AbortSignal reason) belongs to another realm there, so a Proxy of it, or an object created from it, is not instanceof Error and gets no line. In Bun it is and it gets a line. 12 rows of this kind pass on main and fail with this PR. For 3 of them (a Proxy of a DOMException, an object created from one) the read of message throws TypeError: The DOMException.message getter can only be used on instances of DOMException out of the matcher. In the other direction, a Proxy of an error that test code made gains its line. The printer (toMatchSnapshot() of such a value in Jest's format) has had this rule since bun test: Vitest compatibility #44833.
• A monkeypatched Error. 5 probe rows define Error[Symbol.hasInstance] and 2 replace globalThis.Error. Jest follows the hook. Bun does not. 2 of the 7 pass on main.
• More than 1,000 Proxies in a prototype chain are a RangeError. V8 allows 102,400. A Proxy that is its own prototype for 999 calls still gets its line. This is the limit that the printer has: on main, toMatchSnapshot() of a value behind 1,001 Proxy hops throws the same RangeError in a Jest-format file.
• A DOMException as the cause. Jest prints its line from isError() when Node 22 or later runs it. Node 20 prints none, and so does Jest when Bun runs it, because Error.isError(new DOMException()) is false there. Bun's DOMException is not an Error cell (see Error.isError and DOMException #15821, and the open DOMException: make it an ErrorInstance so Error.isError() returns true #47158 and DOMException: inherit from ErrorInstance for [[ErrorData]] and .stack #32898). Until one of those merges, the walk prints the line. After that, is_any_error() prints the same bytes. A DOMException that the runtime made still fails against a Jest-written entry, because the message differs: Node says This operation was aborted, Bun says The operation was aborted.
• A failed require() as the cause. It throws a ResolveMessage, which inherits Error.prototype and is not an Error cell. Its message is Node's text (ResolveMessage::node_message): Cannot find module './x.js', then Require stack: and the absolute path of the importing file. The cause now gets that text as its line, so the entry passes only in the checkout that wrote it. main stores the outer message only.
  • main already stores the same text, with the path, when the ResolveMessage itself is thrown in a Jest-format file.
  • Jest also stores an absolute path when the message of a cause has one. For an fs error it wrote Cause: ENOENT: no such file or directory, open '/tmp/jest-fs/missing.txt'.
  • Jest's own text for a failed require() is Cannot find module './x.js' from 'a.test.js', so no build matches a Jest-written entry here.
  • A portable text would be a change to the message of a failed require() under bun test, for the thrown value and for the cause. This PR does not make it.

Message-only entries

Bun up to 1.4 stores the message of a thrown error and nothing of its causes. It does so in a file with Jest's first line too, where it keeps the first line and appends. #44833 reads such a file in Jest's format. It accepts what an older Bun wrote only when the keys of the file are out of order (may_hold_entries_of_bun, src/runtime/test_runner/snapshot.rs), because Jest keeps its files sorted. A file with one entry, or with entries that Bun appended in order, has no such sign, and #44833 compares it with Jest's text only.

// a.test.js
import { test, expect } from "bun:test";
function E5(m) { this.message = m; }
E5.prototype = Object.create(Error.prototype);
test("t", () => {
  expect(() => { throw new Error("outer", { cause: new E5("inner") }); }).toThrowErrorMatchingSnapshot();
});

// Jest Snapshot v1, https://jestjs.io/docs/snapshot-testing

exports[`t 1`] = `"outer"`;

Bun 1.4.3 writes this .snap when the file holds the first line alone. Bun 1.4.3 and main (2596ac5) pass it. This PR fails it and prints - "outer" against + "outer\nCause: inner". No build rewrites the file.

I had Bun 1.4.3 write 11 such entries into one file, with the keys in order:

• Jest 30.5.2 (Node 26.3.0, --ci) fails the same 7 rows as this PR. For the one-entry file above it prints + Cause: inner.
• With the same entries in a file whose keys are out of order, main and this PR pass all 11 and leave the file alone. A new test in snapshot.test.ts holds this for five kinds of cause, in a Jest file and in a Vitest file.
• A run can turn such a file into a sorted one. I added one new snapshot test beside the 11 and ran bun test without --update-snapshots. All 12 pass, and write_sorted (src/runtime/test_runner/snapshot.rs) writes the file with sorted keys and keeps the 11 old texts. The next run fails 3 of them on main and 7 with this PR. This comes from bun test: Vitest compatibility #44833 and this PR does not change it.
• --update-snapshots rewrites the 7 entries in the text of Jest 30. After that all 11 pass. docs/test/snapshots.mdx now says so.
• Jest 30 writes "outer" too, for the same error without a cause. So the text of an entry does not say whether an older Bun wrote it, or Jest 30 before the code attached a cause.
• Jest made the same change in version 30. Its upgrade guide has a section "Error Causes in Snapshots" and says: "Update any snapshots that fail due to the formatting changes (error causes, empty strings, etc.)."
• I had a second review look at this one question. It found no rule that passes the entry of an older Bun and still fails the entry that Jest 30 fails, so the predicate stays and the rule of bun test: Vitest compatibility #44833 is question 1 above.

Files of Jest 29

I had Jest 29.7.0 write 6 rows (causes: an Error, a string, an ES5 class instance, an object below Error.prototype, a DOMException, a plain object). It stores "outer" for all of them.

So this PR makes Bun agree with Jest 30.5.2 on such a file. The loss against Bun 1.4.3 starts with #44833.

Not changed on purpose

• .resolves.toThrow() (src/runtime/test_runner/expect/toThrow.rs:68) has the same cell-type test from bun test: Vitest compatibility #44833. Jest uses another predicate there: the isError of expect-utils (three Symbol.toStringTag values, or instanceof Error). bun:test: .resolves.toThrow() counts a fulfilled value as thrown only when it is an error #44652 is open for that matcher. Its Expect__isError calls JSObject::defaultHasInstance, which has no limit for Proxies. It can call inherits_error_prototype in its place.
• The Bun-format arm (expect.rs:990) belongs to bun:test: toThrowErrorMatchingSnapshot stores the message of a thrown object that is not an Error #47999.
• jest.rs:706 and :806 (test.each titles) and expect.rs:2873 (print_value) print values. Titles are snapshot keys, so they need their own rows.
• is_string(): git grep -c '\.is_string()' -- src/runtime/test_runner gives 67 on main and 66 here. This PR changes the one that decides a Cause: line. I did not audit the others.
• Other differences in message_with_causes that main has and this PR keeps: a thrown primitive or null stores undefined (Jest 30 throws a TypeError). A message that is not a string is stringified when a cause exists. cause is read without the 'cause' in error test, which matters only for a Proxy whose has and get traps disagree. A cycle of causes stops at the first repeat, and the walk stops after 100 causes (bun test: Vitest compatibility #44833 chose both).
• toBeInstanceOf, toThrow(Class) and expect.any(Class) use the instanceof of JavaScriptCore. For a Proxy that is its own prototype they do not return: expect(proxy).toBeInstanceOf(Error) was still running after 5 s. A Proxy-hop limit in JSObject::defaultHasInstance would end that, and this walk could then become a call of it.

Measurements (linux x64, release builds of main b1305dd and of this PR)

• llvm-size: text 140,315,585 to 140,316,097 (+512). Data and bss do not change. The stripped binary has the same file size.
• llvm-nm -S: SnapshotFormat__kindOf 554 bytes before and after. SnapshotFormat__inheritsErrorPrototype 438 (new). Expect::message_with_causes 778 to 836.
• llvm-objdump of SnapshotFormat__kindOf: 147 instructions in both builds. With call targets normalised, the diff is 0 lines. It executes 105 instructions for a plain object and 136 for a class instance in both builds (gdb).
• Native calls per matcher call, as property reads / isAnyError / the new export (gdb breakpoint counts, slope from 10 to 20 calls): no cause 2/0/0, Error cause 4/1/0, string cause 2/1/0, number and null cause 2/0/0, all equal to main. Plain object 2/1/1 (main 2/1/0). Object below Error.prototype 4/1/1 (main 2/1/0). String object 2/1/1 (main 3/1/0).
• Instructions in message_with_causes with its callees (gdb stepi): no cause 1,408 to 1,408. Error cause 4,738 to 4,739. String cause 3,366 to 3,364. Number and null cause 2,389 to 2,389. Plain object 2,417 to 2,523. Object below Error.prototype 2,417 to 4,703 (it now prints its line). String object 4,681 to 2,554 (it no longer does).
• A Proxy that is its own prototype as the cause: RangeError after 1,000 getPrototypeOf calls, 19 ms on a debug build (main: "outer", 0 calls). Node 26.3.0 throws the same class after 102,400 calls.
• BUN_JSC_validateExceptionChecks=1: 0 reports on the corpus files, the probes and the new test.

Tests

• formats/jest/errors.fixture.js has 21 new tests and inline.fixture.js has 1. Jest 30.5.2 on Node 26.3.0 wrote their 22 snapshots. Node 22.23.4 and 24.22.2 write the same bytes for these two files. The counter in snapshot.test.ts goes from 601 to 623.
• One more test in snapshot.test.ts holds what the corpus cannot: the Cause: line of a failed require() (a ResolveMessage, with its real message), and an inline snapshot that does not match keeps its own message when the getPrototypeOf trap of a cause throws.
• I ran snapshot.test.ts by hand on Linux x64. A release build of main (b1305dd) fails the three jest: corpus tests (15 of 623 corpus tests) and the new test. Debug and release builds of this PR pass them. The no test proof note at the end of this description comes from an automated check that leaves this file to CI.
• Debug build of this PR, also by hand: test/js/bun/test/snapshot-tests/ (196 pass, 1 skip, with FORCE_COLOR=1 as in CI). Also expect.test.js, expect-extend.test.js, expect-deferred.test.ts, pretty-format-overflow.test.ts, pretty-format-overridden-size.test.ts, vitest-flavor.test.ts, jest-extended.test.js, mock-fn.test.js, bun_test.test.ts and printing/ (1976 pass).
• After the merge of main 2596ac5, snapshot.test.ts has 184 tests. A debug build of this PR passes 183 and skips 1. The same build with src/ of main fails 4: the three jest: corpus tests and the require() test.
• On a debug build the six corpus tests take 4 to 5 s of their 5 s limit, on main too, and they time out on a loaded machine. The new rows add about 0.15 s to the three jest: ones. On a release build each takes about 0.3 s.

Designs weighed

• JSValue::is_instance_of with the current Error. It follows Jest in every row (34 of 34). It does not return for a Proxy that is its own prototype, where main answers at once. It also reads globalThis.Error by name for causes that get no line.
• A Proxy-hop limit in JavaScriptCore, then the operator. It changes instanceof for every program, and this fix would wait for a change in another repository.
• A hand-written instanceof that asks the hook and then walks with a limit. It copies the dispatch of JSObject::hasInstance and calls user code from native code.
• The walk of the printer (this PR). bun test: Vitest compatibility #44833 wrote it. It answers for the printer and for the cause loop with one rule and one limit.